Assessing Forecast Accuracy Measures

This paper looks into the issue of evaluating forecast accuracy measures. In the theoretical direction, for comparing two forecasters, only when the errors are stochastically ordered, the ranking of the forecasts is basically independent of the form of the chosen measure. We propose wellmotivated Kullback-Leibler Divergence based accuracy measures. In the empirical direction, we study the performance of several familiar accuracy measures and some new ones in two important aspects: in terms of selecting the known-to-be-better forecaster and the robustness when subject to random disturbance. In addition, our study suggests that, for cross-series comparison of forecasts, individually tailored measures may improve the performance of differentiating between good and poor forecasters.